Deep Equilibrium Multimodal Fusion

Multimodal fusion integrates the complementary information present in multiple modalities and has gained much attention recently. Most existing fusion approaches either learn a fixed fusion strategy during training and inference, or are only capable of fusing the information to a certain extent. Such solutions may fail to fully capture the dynamics of interactions across modalities especially when there are complex intra- and inter-modality correlations to be considered for informative multimodal fusion. In this paper, we propose a novel deep equilibrium (DEQ) method towards multimodal fusion via seeking a fixed point of the dynamic multimodal fusion process and modeling the feature correlations in an adaptive and recursive manner. This new way encodes the rich information within and across modalities thoroughly from low level to high level for efficacious downstream multimodal learning and is readily pluggable to various multimodal frameworks. Extensive experiments on BRCA, MM-IMDB, CMU-MOSI, SUN RGB-D, and VQA-v2 demonstrate the superiority of our DEQ fusion. More remarkably, DEQ fusion consistently achieves state-of-the-art performance on multiple multimodal benchmarks. The code will be released

A Rapid Method for Detection of Salmonella in Milk Based on Extraction of mRNA Using Magnetic Capture Probes and RT-qPCR

Magnetic separation is an efficient method for target enrichment and elimination of inhibitors in the molecular detection systems for foodborne pathogens. In this study, we prepared magnetic capture probes by modifying oligonucleotides complementary to target sequences on the surface of amino-modified silica-coated magnetic nanoparticles and optimized the conditions and parameters of probe synthesis and hybridization. We innovatively put the complexes of magnetic capture probes and target sequences into qPCR without any need for denaturation and purification steps. This strategy can reduce manual steps and save time. We used the magnetic capture probes to separate invA mRNA from Salmonella in artificially contaminated milk samples. The detection sensitivity was 104 CFU/ml, which could be increased to 10 CFU/ml after a 12 h enrichment step. The developed method is robust enough to detect live bacteria in a complex environmental matrix

Learning and Evaluating Human Preferences for Conversational Head Generation

A reliable and comprehensive evaluation metric that aligns with manual preference assessments is crucial for conversational head video synthesis method development. Existing quantitative evaluations often fail to capture the full complexity of human preference, as they only consider limited evaluation dimensions. Qualitative evaluations and user studies offer a solution but are time-consuming and labor-intensive. This limitation hinders the advancement of conversational head generation algorithms and systems. In this paper, we propose a novel learning-based evaluation metric named Preference Score (PS) for fitting human preference according to the quantitative evaluations across different dimensions. PS can serve as a quantitative evaluation without the need for human annotation. Experimental results validate the superiority of Preference Score in aligning with human perception, and also demonstrates robustness and generalizability to unseen data, making it a valuable tool for advancing conversation head generation. We expect this metric could facilitate new advances in conversational head generation